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1. 



DETAILED ACTION 



2. 



Response to Arguments 



3. 35 USC §112 

4. Applicant's arguments, see p9, filed 201 0/08/1 1 , with respect to section 1 1 2 
second paragraph have been fully considered and are persuasive. The section 112 
rejection of claims 1 and 10 have been withdrawn. 

5. 35 USC § 103 

6. Applicant's arguments filed 201 0/08/1 1 have been fully considered but they are 
not persuasive. 

7. At the middle of p1 0, the applicant argues that the prior art does not teach 
video/audio packets synchronized with touch/odor/taste packets and formed into 
frames. However, the examiner respectfully disagrees and argues that Serbenescu 
teaches audio/video/touch/odor/taste are synchronized to provide a more immersive 
multimedia experience ([0005, 0008, 0030, 0059-0061]). Also, the new prior art 
Murakami teaches audio / video packets having been placed into frames (fig.1, fig.3; 
8:1-15; 8:62-9:8). It would have been obvious to one of ordinary skill in the art at the 
time of the invention to place audio/video packets in frames along with touch/odor/taste 
packets to provide an efficient multiplexing process and to allow for media to be 
transported on different transmission means (fig.1, fig.3; 8:1-15; 8:62-9:8). 
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8. At the bottom of p1 0, the applicant argues that Konrad does not teach sensory 
data being formed into packets. However, the examiner respectfully disagrees and 
argues that packet formation can be inferred because the reference teaches packet 
switching (9:1-9:8) and it is necessary for the sensory data to be transported across a 
network. 

9. Claim Rejections - 35 USC § 103 

1 0. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

11. A patent may not be obtained though the invention is not identically disclosed or described as set forth 
in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art 
are such that the subject matter as a whole would have been obvious at the time the invention was made to a 
person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 

12. Claim 1, 7, 9, 10, 15 rejected under 35 U.S.C. 103(a) as being unpatentable 
over Konrad (US-5974444), and further in view of Shibata (US-5402418), Tremblay 
(US-6275213), Wittek (US-5963302), Rowan (US-5341229), Serbanescu (US- 
20030162595), Murakami (US-5930251). 

1 3. As to claim 1 , 7, 9, 1 0, 1 5: Konrad teaches a system for providing actual-feeling 
multimedia data, comprising: a video/audio data generating means for generating 
video/audio data by receiving multimedia data from an external device (fig.3c; col.9, 
ln.13-59; col.13, ln.44-50; col.21, ln.1-44 ESP col.9, ln.33-35, 50-52: visual display 
and audio output); a touch data describing ... expressed in the multimedia data 
transmitted from the external device based on a predefined touch data descriptor 
(fig.3c; col.9, ln.13-59; col.13, ln.44-50; col.21, ln.1-44 ESP col.9, ln.37-38, 52-53: 
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tactile output devices; wherein data descriptors necessary in order to output 
something); an odor data describing means for describing an odor expressed in the 
multimedia data received from the external device based on a predefined odor data 
descriptor (fig.3c; col.9, ln.13-59; col.13, ln.44-50; col.21, ln.1-44 ESP col.9, ln.36, 
57: olfactory output devices; wherein data descriptors necessary in order to 
output something); a taste data describing means for describing a taste expressed in 
the multimedia data received from the external device based on a predefined taste data 
descriptor (fig.3c; col.9, ln.13-59; col.13, ln.44-50; col.21, ln.1-44 ESP col.9, ln.36, 
57: gustatory output devices; wherein data descriptors necessary in order to 
output something); a video/audio packet forming means for forming video/audio 
packets out of the video/audio data generated in the video/audio generating means 
(fig. 3c, 108; col.8, ln.65-col.9, ln.8: audio and video data must be packetized in 
order to be transported over the network from the remote host to the local host); 
a touch/odor/taste packet forming means for forming a touch packet, an odor packet, 
and a taste packet out of the touch, odor and taste data described in the touch data 
describing means, the odor data describing means, and the taste data describing 
means, respectively (fig.3c, 108; col.8, ln.65-col.9, ln.8: touch, odor, and taste data 
must be packetized in order to be transported over the network from the remote 
host to the local host); a multiplexing means for multiplexing the video/audio packets 
generated in the video/audio packet generating means and at least one of the touch 
packet, the odor packet and the taste packet formed in the touch/odor/taste packet 
forming means (fig.3c; col.7, In. 16-19, 39-45; col. 10, In. 12-18: a server performs 
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multiplexing and de-multiplexing of data) ... a transmitting means for transmitting a 
multiplexed and synchronized packet obtained in the multiplexing means (fig.3c; col.7, 
In. 16-19, 39-45; col. 10, In. 12-18: a server performs multiplexing and de- 
multiplexing of data and transmits and receives data); a receiving means for 
receiving said multimedia data ... (fig.3c; col.7, In. 16-19, 39-45; col. 10, In. 12-18: a 
server performs multiplexing and de-multiplexing of data and transmits and 
receives data); a demultiplexing means for demultiplexing the multiplexed and 
synchronized packet received by the receiving means into the video data, the audio 
data, the touch data, the odor data and the taste data (fig.3c; col.7, In. 16-19, 39-45; 
col. 10, In. 12-18: a server performs multiplexing and de-multiplexing of data). 

14. Konrad may not explicitly teach to thereby synchronize the video/audio packets 
with at least one of the touch/odor/taste packets, wherein the multiplexing means adds 
to at least one of the touch/odor/taste packets formed in the touch/odor/taste packet 
forming means to a plurality of video/audio packets generated in the video/audio 
generating means to form a plurality of multimedia data frames to thereby multiplex and 
synchronize the video/audio packets with at least one of the touch/odor/taste packets. 
However, Serbanescu teaches synchronizing various output media, such as taste, odor, 
touch along with audio and video to have a rich, interactive multimedia experience 
having all the senses engaged ([0005, 0008, 0030, 0059-0061]). In addition, Murakami 
teaches audio, video, and data packets being multiplexed and then placed into a fixed- 
length packet (i.e. transmission frame) (fig.1, fig.3; 8:1-15; 8:62-9:8). It would have 
been obvious to one of ordinary skill at the time of the invention to combine video/audio 
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packets with touch/odor/taste packets and place them into a multimedia data frame as 
stated in the claim for the purposes of synchronizing the output of the video/audio and 
touch/odor/taste packets and present a seamless multimedia experience so that all 
sense data appears at the time that they are supposed to. Serbanescu teaches that it is 
desirable to synchronize audio/video/taste/odor/touch data while Murakami teaches 
multimedia variable length packets (8:19) are placed into transmission frames (fig.3). It 
would have been obvious to combine Murakami and Serbanescu with Konrad because 
Konrad suggests video, audio, touch, odor, taste data multiplexed and presented to a 
user for an interactive multimedia experience and Murakami and Serbanescu together 
suggest video, audio, touch, odor, taste data multiplexed, synchronized together, and 
placed into a transmission frame to present a seamless multimedia experience as well 
as to provide an efficient multiplexing process and to allow for media to be transported 
on different transmission means (5:5-11). 

1 5. Konrad may not explicitly teach a video device for decoding and outputting the 
video data demultiplexed by the demultiplexing means; an audio device for decoding 
and outputting the audio data demultiplexed by the demultiplexing means. However, 
Shibata teaches a video device for decoding and outputting the video data 
demultiplexed by the demultiplexing means (col.3, ln.58 - col.4, In. 12: audio and 
video decoder and de-multiplexer for output); an audio device for decoding and 
outputting the audio data demultiplexed by the demultiplexing means (col.3, ln.58 - 
col.4, In. 12: audio and video decoder and de-multiplexer for output). 
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16. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of Shibata into Konrad since Konrad 
suggests a human interface service that receives and outputs audio and video data 
(fig.3c; col.9, ln.13-59; col.13, ln.44-50; col.21, ln.1-44 ESP col.9, ln.33-35, 50-52: 
visual display and audio output) in general and Shibata suggests decoding and de- 
multiplexing audio and video data, the motivation being to play audio and display video 
for a video conference call (col.3, In. 58 - col.4, In. 12). 

17. Konrad may not explicitly teach a vibration device for providing vibration to a user 
by interpreting the touch data demultiplexed by the demultiplexing means. However, 
Tremblay teaches a vibration device for providing vibration to a user by interpreting the 
touch data demultiplexed by the demultiplexing means (col.1, In. 15-39; col.2, In. 14-33: 
vibration from tactile information). 

18. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of Tremblay into Konrad since Konrad 
suggests a human interface service receiving and demultiplexing tactile data and 
outputting through a tactile output device (col.9, ln.29-59) in general and Tremblay 
suggests a tactile output device producing vibrations, the motivation being to provide a 
virtual reality experience for the user (col.1, In. 15-39, 40-52; col.2, In. 14-33). 

1 9. Konrad may not explicitly teach an odor device for spraying chemical aromatics 
to a user by interpreting the odor data demultiplexed by the demultiplexing means. 
However, Wittek teaches an odor device for spraying chemical aromatics to a user by 



Application/Control Number: 10/579,349 Page 8 

Art Unit: 2466 

interpreting the odor data demultiplexed by the demultiplexing means; and (col.2, ln.50- 
65; col.3, ln.33-45: scent control signal for spaying chemical aromas). 

20. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of Wittek into Konrad since Konrad 
suggests a human interface service receiving and demultiplexing olfactory data and 
outputting through an olfactory output device (col.9, ln.57) in general and Wittek 
suggests a olfactory output device producing chemical aromatics, the motivation being 
to intensify visual and acoustic reception (col.1, ln.22-25). 

21 . Konrad may not explicitly teach and a taste device for releasing a taste forming 
material to a user by interpreting the taste data demultiplexed by the demultiplexing 
means. However, Rowan teaches a taste device for releasing a taste forming material 
to a user by interpreting the taste data demultiplexed by the demultiplexing means 
(col.1, In. 14-33: chemical release mechanism for taste in order to enhance audio 
or visual stimuli). 

22. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of Rowan into Konrad since Konrad 
suggests a human interface service receiving gustatory data and outputting through a 
gustatory output device (col.9, ln.39-40) in general and Rowan suggests a gustatory 
output device releasing chemicals for taste, the motivation being to enhance audio and 
visual stimuli (col.1, In. 14-33). 
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23. Claim 5 rejected under 35 U.S.C. 103(a) as being unpatentable over Konrad 
(US-5974444), Shibata (US-5402418), Tremblay (US-6275213), Wittek (US-5963302), 
Rowan (US-5341229), Serbanescu (US-20030162595), Murakami (US-5930251) as 
applied to claim 1 above, and further in view of Wan (US-20020054608). 

24. As to claim 5: Konrad teaches the apparatus as recited in claim 1 , wherein the 
touch/odor/taste packet forming means forms a touch packet including information on 
whether the touch data are described, ... , and information on the touch data descriptors 
described in the touch data describing means (fig. 3c; col.9, In. 13-59; col. 13, ln.44-50; 
col.21, In. 1-44: information packets describing touch); an odor packet including 
information on whether odor data are described, ... , and information on the odor data 
descriptors described in the odor data describing means (fig.3c; col.9, In. 13-59; col. 13, 
ln.44-50; col.21, In. 1-44: information packets describing odor); and a taste packet 
including information on whether taste data are described, ... , and information on the 
taste data descriptors described in the taste data describing means (fig.3c; col.9, In. 13- 
59; col. 13, ln.44-50; col.21, In. 1-44: information packets describing taste). 

25. Konrad may not explicitly teach information on touch packet length ... information 
on an odor packet length ... information on a taste packet length. 

26. However, information describing the length of a packet is commonly known in the 
art. This is illustrated by Wan where a video / audio packet includes information with 
regards to packet length ([0014]). Thus, it would have been obvious to one of ordinary 
skill in the art at the time of the invention to implement packet lengths in odor, taste, and 
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touch packets in order to know their start and ending point of the packets and to 
accurately process them. 

27. Claim 2, 3, 4, 8 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Konrad (US-5974444), Shibata (US-5402418), Tremblay (US-6275213), Wittek (US- 
5963302), Rowan (US-5341229), Serbanescu (US-200301 62595), Murakami (US- 
5930251) as applied to claim 1, 7 above, and further in view of Roy (US- 
20010036868), DiNunzio (US-6007338). 

28. As to claim 2: Konrad teaches the apparatus as recited in claim 1 . 

29. Konrad may not explicitly teach wherein the touch data describing means 
describes vibration expressed in the multimedia data transmitted from the external 
device based on a descriptor describing whether touch data are described; a descriptor 
describing whether right/left movement is described; a descriptor describing whether 
up/down movement is described; a descriptor describing whether back/forth movement 
is described. However, Roy teaches wherein the touch data describing means 
describes vibration expressed in the multimedia data transmitted from the external 
device based on a descriptor describing whether touch data are described; a descriptor 
describing whether right/left movement is described ([0014], [0056]; claim 16: axial 
displacement for tilt); a descriptor describing whether up/down movement is described 
([0048], [0051-0052], [0056]: raise and lower vertically); a descriptor describing 
whether back/forth movement is described ([0014], [0056]; claim 16: axial 
displacement for tilt). 
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30. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of Roy into Konrad since Konrad 
suggests a human interface service where tactile data having descriptors is conveyed 
over a network and output through a tactile output device such as a pressure suit (col.9, 
ln.53-54) in general and Roy suggests a tactile output device having axial displacement 
for front / back, left / right tilt and a vertical up / down motion controlled by control 
signals, the motivation being to enhance an audiovisual experience, such as gaming or 
movies, with motion transducers ([0009-0011]). 

31 . Konrad, Roy may not explicitly teach a descriptor describing a distance of 
movement; a descriptor describing a speed of movement; a descriptor describing an 
acceleration of movement. However, Roy teaches control values for the actuators and 
the possibility of precisely controlling the mechanical movements transmitted to the 
chair adapter ([0014], [0056]). It would have been obvious to one of skill in the art at 
the time of the invention to precisely control the motion transducers with regards to 
distance, speed, and acceleration in order to customize the vertical and axial motions 
and not be limited to a single setting with regards to distance, speed, and acceleration. 
This allows the motion transducers to be customized to a plethora of situations, from the 
quick vibrating hum of aircraft to the slow rolling waves of the ocean rather than being 
confined to a single setting. 

32. Konrad, Roy may not explicitly teach a descriptor describing whether right/left 
rotation is described; a descriptor describing an angle of right/left rotation; a descriptor 
describing a speed of right/left rotation; and a descriptor describing an acceleration of 
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right/left rotation. However, DiNunzio teaches a descriptor describing whether right/left 
rotation is described (fig.4a, 4b; col.8, ln.48-67: yaw rotation around the z-axis); a 
descriptor describing an angle of right/left rotation (fig.4a, 4b; col.8, ln.48-67: yaw 
rotation around the z-axis has degree of motion less than 90 degrees + / -). 

33. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of DiNunzio into Roy since Roy 
suggests a tactile output device having axial displacement for front / back, left / right tilt 
and a vertical up / down motion with motion transducers controlled by control signals 
([0009-0011]) in general and DiNunzio suggests a tactile output device having yaw, roll 
and pitch motions synchronized with audio and video the motivation being to provide for 
greater degrees of freedom and offer more motions with which to enhance an 
audiovisual experience (fig.4a, 4b; col.8, ln.39-67; claim 2). 

34. Konrad, DiNunzio may not explicitly teach a descriptor describing a speed of 
right/left rotation; and a descriptor describing an acceleration of right/left rotation. 
However, DiNunzio teaches controlling the speed of the overall experience, make-up of 
the roller coaster track, and simulates the acceleration experienced by the user (col.5, 
In. 1-8; col.6, In. 1-14; col.9, In. 1-23). In order to simulate speed and acceleration 
around the roller coaster track, acceleration and speed control signals / descriptors for 
the motion transducers are necessary. It would have been obvious to one of ordinary 
skill in the art to implement speed and acceleration descriptors in DiNunzio in order to 
make feasible the custom design of roller coaster elements with loops and corkscrews 
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of varying diameter and intensity and provide a varied experience as opposed to having 
the exact same speed and acceleration throughout the entire simulation. 

35. As to claim 3: Konrad teaches the apparatus as recited in claim 2. 

36. Konrad may not explicitly teach wherein the odor data describing means 
describes an odor expressed in the multimedia data transmitted from the external 
device based on a descriptor describing whether the odor data are described; a 
descriptor describing a kind of the odor; and a descriptor describing an intensity of the 
odor. However, Wittek teaches wherein the odor data describing means describes an 
odor expressed in the multimedia data transmitted from the external device based on a 
descriptor describing whether the odor data are described; a descriptor describing a 
kind of the odor; and a descriptor describing an intensity of the odor (col.3, ln.32-45: 
control signals indicating the type and intensity of the scent). 

37. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of Wittek into Konrad since Konrad 
suggests a human interface service multiplexing olfactory data having descriptors and 
output through an olfactory I/O device (col.9, ln.57) in general and Wittek suggests 
scent release mechanisms synchronized with audiovisual affects where control signals 
control the type and intensity of the scent, the motivation being to intensify and enrich 
the audiovisual experience (col.1, ln.22-25). 
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38. As to claim 4: Konrad teaches the apparatus as recited in claim 3. 

39. Konrad may not explicitly teach wherein the taste data describing means 
describes a taste expressed in the multimedia data transmitted from the external device 
based on a descriptor describing whether the taste data are described; a descriptor 
describing a kind of the taste; and a descriptor describing an intensity of the taste. 

40. However, Konrad teaches a human interface service receiving multiplexed 
gustatory signals over a network for output to a gustatory I/O device (col.9, ln.57). 
Wittek teaches control signals for controlling the type and intensity of the scent (col.3, 
ln.32-45). It would have been obvious to one of ordinary skill in the art at the time of the 
invention to use control signals similar to those of Wittek and apply them to the 
gustatory output of Konrad, the motivation being to intensify and enrich the audiovisual 
experience with taste sensations and to be able to customize the taste sensations to 
different audiovisual situations presented (Wittek, col.1, ln.22-25). 

41 . As to claim 8: Konrad teaches the method as recited in claim 7. 

42. Konrad may not explicitly teach wherein in the step b), the vibration expressed in 
the multimedia data transmitted from the external device is described based on a 
descriptor describing whether touch data are described; A descriptor describing whether 
right/left movement is described; a descriptor describing whether up/down movement is 
described; a descriptor describing whether back/forth movement is described. 

43. Roy teaches wherein in the step b), the vibration expressed in the multimedia 
data transmitted from the external device is described based on a descriptor describing 
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whether touch data are described; A descriptor describing whether right/left movement 
is described ([0014], [0056]; claim 16: axial displacement for tilt); a descriptor 
describing whether up/down movement is described ([0048], [0051-0052], [0056]: raise 
and lower vertically); a descriptor describing whether back/forth movement is 
described ([0014], [0056]; claim 16: axial displacement for tilt). 

44. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of Roy into Konrad since Konrad 
suggests a human interface service where tactile data having descriptors is conveyed 
over a network and output through a tactile output device such as a pressure suit (col.9, 
In. 53-54) in general and Roy suggests a tactile output device having axial displacement 
for front / back, left / right tilt and a vertical up / down motion controlled by control 
signals, the motivation being to enhance an audiovisual experience, such as gaming or 
movies, with motion transducers ([0009-0011]). 

45. Konrad, Roy may not explicitly teach a descriptor describing a distance of 
movement; a descriptor describing a speed of movement; a descriptor describing an 
acceleration of movement. 

46. However, Roy teaches control values for the actuators and the possibility of 
precisely controlling the mechanical movements transmitted to the chair adapter 
([0014], [0056]). It would have been obvious to one of skill in the art at the time of the 
invention to precisely control the motion transducers with regards to distance, speed, 
and acceleration in order to customize the vertical and axial motions and not be limited 
to a single setting with regards to distance, speed, and acceleration. This allows the 
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motion transducers to be customized to a plethora of situations, from the quick vibrating 
hum of aircraft to the slow rolling waves of the ocean rather than being confined to a 
single setting. 

47. Konrad, Roy may not explicitly teach a descriptor describing whether right/left 
rotation is described; a descriptor describing an angle of right/left rotation; a descriptor 
describing a speed of right/left rotation; a descriptor describing an acceleration of 
right/left rotation. However, DiNunzio teaches a descriptor describing whether right/left 
rotation is described (fig.4a, 4b; col.8, ln.48-67: yaw rotation around the z-axis); a 
descriptor describing an angle of right/left rotation (fig.4a, 4b; col.8, ln.48-67: yaw 
rotation around the z-axis has degree of motion less than 90 degrees + / -). 

48. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of DiNunzio into Roy since Roy 
suggests a tactile output device having axial displacement for front / back, left / right tilt 
and a vertical up / down motion with motion transducers controlled by control signals 
([0009-0011]) in general and DiNunzio suggests a tactile output device having yaw, roll 
and pitch motions synchronized with audio and video the motivation being to provide for 
greater degrees of freedom and offer more motions with which to enhance an 
audiovisual experience (fig.4a, 4b; col.8, ln.39-67; claim 2). 

49. Konrad, DiNunzio may not explicitly teach a descriptor describing a speed of 
right/left rotation; a descriptor describing an acceleration of right/left rotation. 

50. However, DiNunzio teaches controlling the speed of the overall experience, 
make-up of the roller coaster track, and simulates the acceleration experienced by the 
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user(col.5, In. 1-8; col.6, In. 1-14; col.9, In. 1-23). In order to simulate speed and 
acceleration around the roller coaster track, acceleration and speed control signals / 
descriptors for the motion transducers are necessary. It would have been obvious to 
one of ordinary skill in the art to implement speed and acceleration descriptors in 
DiNunzio in order to make feasible the custom design of roller coaster elements with 
loops and corkscrews of varying diameter and intensity and provide a varied experience 
as opposed to having the exact same speed and acceleration throughout the entire 
simulation. 

51 . Konrad, Roy, DiNunzio may not explicitly teach the odor expressed in the 
multimedia data received from the external device is described based on a descriptor 
describing whether the odor data are described; a descriptor describing a kind of the 
odor; and a descriptor describing an intensity of the odor. 

52. Wittek teaches the odor expressed in the multimedia data received from the 
external device is described based on a descriptor describing whether the odor data are 
described; a descriptor describing a kind of the odor; and a descriptor describing an 
intensity of the odor (col.3, ln.32-45: control signals indicating the type and 
intensity of the scent). 

53. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of Wittek into Konrad since Konrad 
suggests a human interface service multiplexing olfactory data having descriptors and 
output through an olfactory I/O device (col.9, In. 57) in general and Wittek suggests 
scent release mechanisms synchronized with audiovisual affects where control signals 
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control the type and intensity of the scent, the motivation being to intensify and enrich 
the audiovisual experience (col.1, ln.22-25). 

54. Konrad, Roy, DiNunzio, Wittek may not explicitly teach and, the taste expressed 
in the multimedia data received from the external device is described based on a 
descriptor describing whether the taste data are described; a descriptor describing a 
kind of the taste; and a descriptor describing an intensity of the taste. 

55. However, Konrad teaches a human interface service receiving multiplexed 
gustatory signals over a network for output to a gustatory I/O device (col.9, ln.57). 
Wittek teaches control signals for controlling the type and intensity of the scent (col.3, 
ln.32-45). It would have been obvious to one of ordinary skill in the art at the time of the 
invention to use control signals similar to those of Wittek and apply them to the 
gustatory output of Konrad, the motivation being to intensify and enrich the audiovisual 
experience with taste sensations and to be able to customize the taste sensations to 
different audiovisual situations presented (Wittek, col.1, ln.22-25). 

56. Claim 11 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Konrad (US-5974444), Shibata (US-5402418), Tremblay (US-6275213), Wittek (US- 
5963302), Rowan (US-5341229), Serbanescu (US-200301 62595), Murakami (US- 
5930251) as applied to claim 10 above, and further in view of Farmer (US- 
5907366). 

57. As to claim 1 1 : Konrad teaches the system as recited in claim 10. 
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58. Konrad may not explicitly teach wherein the demultiplexing means deletes 
network-related information from the received packet in form of a compressed stream 
by depacketizing divides the depacketized packet into the video data, the audio data ... 
on a basis of a multimedia data frame, and transmits the video data, the audio data ... 
to corresponding devices based on header information. 

59. Farmer teaches wherein the demultiplexing means deletes network-related 
information from the received packet in form of a compressed stream by depacketizing 
(col.9, In. 15-40: transport de-multiplexer separates signal and removes transport 
header information) divides the depacketized packet into the video data, the audio 
data (col.9, In. 15-40: transport de-multiplexer separates signal into audio and 
video data), ... on a basis of a multimedia data frame (col.9, In. 15-40: MPEG frame), 
and transmits the video data, the audio data (col.9, In. 15-40: audio / video payload 
routed to storage), ... to corresponding devices based on header information (col.9, 
In. 15-40: header information indicates type of payload). 

60. Farmer may not explicitly teach divides the depacketized packet into ... the touch 
data, the odor data and the taste data on a basis of a multimedia data frame, and 
transmits the ... touch data, the odor data and the taste data to corresponding devices 
based on header information. 

61 . However, Konrad teaches the multiplexing and de-multiplexing of audio, video, 
touch, smell, and taste information (fig.3c; col.9, In. 13-59; col. 13, ln.44-50; col.21, 

In. 1-44) and Farmer teaches all in the above paragraph with regards to audio and video 
packets. It would have been obvious to one of ordinary skill in the art at the time of the 
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invention to combine Farmer and Konrad and implement the de-multiplexing, de- 
packetization and transmission of touch, odor, and taste data based on the header in 
order to separate the different payloads and process them with the appropriate output 
devices and create an interactive multimedia experience involving all of the senses. 

62. Claim 12, 13, 14 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Konrad (US-5974444), Shibata (US-5402418), Tremblay (US-6275213), Wittek (US- 
5963302), Rowan (US-5341229), Serbanescu (US-200301 62595), Murakami (US- 
5930251) as applied to claim 10 above, and further in view of Roy (US- 
20010036868), Ro (US-20020150123). 

63. As to claim 12: Konrad teaches the system as recited in claim 10, ... which are 
demultiplexed in the demultiplexing means (fig.3c; col.7, In. 16-19, 39-45; col. 10, In. 12- 
18: a server performs multiplexing and de-multiplexing of data), based on a 
predefined touch data descriptor (col.9, ln.80: tactile output devices; wherein touch 
data descriptors necessary for proper functionality). 

64. Konrad may not explicitly teach wherein the vibration device moves to right and 
left, back and forth, and up and down or rotates by interpreting the touch data, ... and a 
starting time and a duration time of movement or rotation operation. However, Roy 
teaches wherein the vibration device moves to right and left, back and forth ([0014], 
[0056]; claim 16: axial displacement for tilt), and up and down or rotates by 
interpreting the touch data ([0048], [0051-0052], [0056]: raise and lower vertically), . .. 
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and a starting time and a duration time of movement or rotation operation ([0042]: start 
and duration time for transducer action). 

65. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of Roy into Konrad since Konrad 
suggests a human interface service where tactile data having descriptors is conveyed 
over a network and output through a tactile output device such as a pressure suit (col.9, 
ln.53-54) in general and Roy suggests a tactile output device having axial displacement 
for front / back, left / right tilt and a vertical up / down motion controlled by control 
signals, the motivation being to enhance an audiovisual experience, such as gaming or 
movies, with motion transducers ([0009-0011]). 

66. Konrad, Roy may not explicitly teach are synchronized with a moving picture and 
a sound outputted from the video device and the audio device, respectively ([0005], 
[0008], [0030], [0059-0061]). However, Serbanescu teaches synchronizing various 
output media, such as taste, odor, touch along with audio and video ([0005], [0008], 
[0030], [0059-0061]). Serabanescu may not explicitly teach that the packets 
themselves are synchronized. However, it is common knowledge in the art that 
synchronization of packets is a technique for having output media match-up for 
presentation. This is illustrated by Ro who teaches audio and video packets 
synchronized together in order to present the audio and video together (Ro; [0045], 
[0055], [0064]). 

67. Thus, it would have been obvious to one of ordinary skill in the art to implement 
the teachings of Serabanescu into Konrad since Konrad teaches a human interface 
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service outputting audio, video, taste, smell, and touch information to a user (col.9, 
In. 13-59; col. 13, ln.44-50; col.21, In. 1-44) and Serabanescu suggests synchronization 
between audio, video, taste, smell, and touch in order to present an interactive media 
experience ([0005], [0008], [0030], [0059-0061]). 

68. As to claim 13: Konrad teaches the system as recited in claim 12, ... which are 
demultiplexed in the demultiplexing means (fig. 3c; col.7, In. 16-19, 39-45; col. 10, In. 12- 
18: a server performs multiplexing and de-multiplexing of data), based on a 
predetermined odor data descriptor (col.9, ln.57: olfactory output devices; wherein 
olfactory data descriptors necessary for proper functionality). 

69. Konrad may not explicitly teach wherein the odor device sprays the chemical 
aromatics by interpreting the odor data ... and a starting time and a duration time of 
spraying operation are synchronized with a moving picture and a sound outputted from 
the video device and the audio device, respectively. However, Wittek teaches wherein 
the odor device sprays the chemical aromatics by interpreting the odor data (col.2, 

In. 50-65; col. 3, In. 33-45: scent control signal for spaying chemical aromas) ... and 
a starting time and a duration time of spraying operation are synchronized with a 
moving picture and a sound outputted from the video device and the audio device, 
respectively (col. 14, 5-59: scent entrance and duration is synchronized with 
movie). 

70. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of Wittek into Konrad since Konrad 
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suggests a human interface service multiplexing olfactory data having descriptors and 
output through an olfactory I/O device (col.9, ln.57) in general and Wittek suggests 
scent release mechanisms synchronized with audiovisual affects where control signals 
control the type and intensity of the scent, the motivation being to intensify and enrich 
the audiovisual experience (col.1, ln.22-25). 

71 . As to claim 14: Konrad teaches the system as recited in claim 13, ... which are 
demultiplexed in the demultiplexing means (fig.3c; col.7, In. 16-19, 39-45; col. 10, In. 12- 
18: a server performs multiplexing and de-multiplexing of data), based on a 
predetermined taste data descriptor (col.9, ln.57: gustatory output devices; wherein 
gustatory data descriptors necessary for proper functionality). 

72. Konrad may not explicitly teach wherein the taste device releases taste forming 
materials by interpreting the taste data, ... and a starting time and a duration time of 
releasing operation are synchronized with a moving picture and a sound outputted from 
the video device and the audio device, respectively. However, Rowan teaches wherein 
the taste device releases taste forming materials by interpreting the taste data (col.1, 
In. 14-33: chemical release mechanism for taste in order to enhance audio or 
visual stimuli), ... and a starting time and a duration time of releasing operation are 
synchronized with a moving picture and a sound outputted from the video device and 
the audio device, respectively (col.1, In. 14-33: chemical release mechanism 
synchronized with audiovisual presentation). 
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73. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of Rowan into Konrad since Konrad 
suggests a human interface service receiving gustatory data and outputting through a 
gustatory output device (col.9, ln.39-40) in general and Rowan suggests a gustatory 
output device releasing chemicals for taste, the motivation being to enhance audio and 
visual stimuli (col.1, In. 14-33). 



74. Conclusion 

75. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See M PEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

76. A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 



Application/Control Number: 10/579,349 Page 25 

Art Unit: 2466 

77. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ANDREW OH whose telephone number is (571)270- 
5273. The examiner can normally be reached on M-F 8:30AM - 5AM EST. 

78. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Daniel J. Ryman can be reached on (571)272-3152. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

79. Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

80. 81. 

82. /A. O./ 

83. Examiner, Art Unit 2466 



/Daniel J. Ryman/ 

Supervisory Patent Examiner, Art Unit 2466 



